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This booklet is intended as a general 
guide for researchers who have 
decided to use a mail questionnaire to 
collect data. Seven methods which 
were reported in the literature as suc- 
cessful in improving response rates 
are outlined here. Any combination of 
these methods may also be used de- 
pending on the resources available for 
each specific survey. 


The material presented here does not 
evaluate or compare mail surveys with 
other alternatives (i.e. telephone or 
personal interviews). It reflects the 
literature available on this topic and 

is not intended to be taken as an offi- 
cial Statistics Canada viewpoint. A 
bibliography is included for each of 
the methods. 
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|. ADVANCE NOTIFICATIO 


There is general agreement among 
researchers that contacting respond- 
ents prior to sending them the mail 
questionnaire increases the number of 
questionnaires returned. It does not, 
however, seem to affect how quickly 
questionnaires are returned, how many 
of the questions are completed and in 
how much detail each question is 
answered. 


Advance notification allows the re- 
searcher to identify himself, to discuss 
the purpose of the study and to request 
the cooperation of the respondent. 
The main advantage of this pre-con- 
tact seems to be that the respondent 
can be alerted for the arrival of the 
questionnaire and is not as likely to 
discard it as householder/ junk mail. 
Forms of pre-contact that have been 
found to be effective are: letter; post- 
card; telephone call; and personal 
Visit. 


2. ANONYMITY OF RESPONDENTS 


A researcher may feel that the topical 
nature of some questions necessitates 
protecting the anonymity of the re- 
spondent if the responses are to be 
truly candid. However, maintaining 
the anonymity of respondents ina 
mail survey has not been shown to 
generally improve response rates. 
Studies show that there is no signifi- 
cant difference in desire for anonymity 
between those respondents with posi- 
tive opinions and those respondents 
with negative opinions about the sub- 
ject matter of the questionnaire. 
Whether or not the individual prefers 
anonymity seems to depend on his 
perception of himself and the level of 
his confidence in his opinions. 


Anonymity might be considered for 
the questionnaire which asks for de- 
tails concerning personal finances. In 
these instances a pre-test may indicate 
whether anonymity of respondents is 
necessary or whether assurances of 
confidentiality are likely to suffice. 


2.1 PERSONALIZATION 


Two types of personalization have 
been discussed in the literature. One 
type concerned the use of individual 
names and addresses on the survey 
material sent to respondents. Personal- 
ization of this type can reduce the 
respondent’s sense of anonymity. Per- 
sonal salutations and individually ad- 
dressed envelopes and covering letters 
which have been tested with mail sur- 
veys show inconclusive results. In 
some of the surveys reviewed the re- 
sponse rates were actually reduced. 
This was particularly true in cases 
where the respondent felt his responses 
were no longer anonymous. 


The other way that a survey can be 
personalized is to have the survey 
sponsor add as a personal toucha 
handwritten signature or short hand- 
written postscript to the questionnaire 
or covering letter. This does not re- 
quire a substantial amount of manual 
labour for the sponsor and has been 
shown to increase response rates. 


3.COVERING LETTER 


A covering letter should always be a 
part of the survey package sent to re- 
spondents since it has been shown to 
have a positive effect on response 
rates. It should be long enough to in- 
clude all points that are relevant to 
motivating the respondent and assur- 
ing confidentiality of responses. It 
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should also introduce and personalize 
the researcher, identify the sponsoring 
organization and outline the objec- 
tives of the survey. 


3.1 TYPE OF APPEAL 


Some researchers have experimented 
with different types of appeals to re- 
spondents in the covering letter. This 
experience indicates that appeals to 
the respondent based on the social 
benefits of the study, appeals to help 
the researcher with his study and more 
general appeals to the altruism of the 
respondent are not particularly effec- 
tive in improving response rates and 
could be deleted from a covering 
letter. 


3.2 SURVEY SPONSORSHIP & 
METHOD OF SIGNING 


The organization that appears on the 
letterhead of the covering letter and 
the title of the person signing the let- 
ter can affect the return rate. In one 
study, it was shown that the response 
was higher for government sponsor- 
ship than it was for either commercial 
or university sponsorship. With re- 
spect to titles, it seems that if the per- 
son who signs the letter is well-known 
or if his position seems to be impor- 
tant, the returns are slightly higher. 


3.0 FORMAT 


Covering letters are usually separate 
from the questionnaire but some stud- 
ies where the survey material was brief 
have combined the two and exper- 
ienced no detrimental results. 


3.4 DEADLINES 


Few studies have effectively tested the 
effect of deadlines on return rates. A 
general opinion, however, seems to be 
that deadlines do not increase response 
rates but do increase early response. 


4. INCENTIVES 


A number of studies have researched 
the effects on response rates of includ- 
ing incentives with mail question- 
naires. All have reported favourable 
results. Cash rewards in any amount 
significantly increase responses, al- 
though other types of premiums have 
also been shown to have positive ef- 
fects. However, it has been demon- 
strated that the inclusion of an incen- 
tive (either money or other premium) 
results in much higher return rates 
than the promise of an incentive on 
return of the questionnaire. The prom- 
ise of an incentive has been shown to 
result in only marginally better rates. 
A note giving a reasonable explana- 
tion for the inclusion or the promise 
of the incentive should be included in 
the covering letter. 


Although inclusion of an incentive 
has a greater initial cost the higher re- 
turn rates reduce the unit costs. Vari- 
ous incentives which have been used 
in studies and which resulted in better 
than expected return rates are: 
trading stamps 

unused postage stamps 

packets of stamps for collectors 

letter openers 

tie clips 

diaries 

books 

memorandum books 

pencils, pens 


draws or lotteries (a turkey) 
plastic identification cards 
money (domestic and foreign) 


0.POSTAL DETAILS & ADDRESSING 


The methods of posting and addressing 
to be used in mail surveys are worth 
considering since they do affect re- 
sponse rates. 


Every researcher who is intending to 
use mail questionnaires as the survey 
vehicle should include in the budget 
details of the cost for return addressed 
stamped envelopes. When such en- 
velopes have not been included in the 
package sent to respondents, the 
return rates have been very low. 


Studies conducted with various types 
of postal arrangements on both out- 
going and return envelopes have 
shown that any one of the following 
conditions usually results in higher 
return rates: 


1) the use of special delivery or air- 
mail rather than first or third class 
postage; 

2) the use of hand stamped rather 
than metered envelopes; 

3) the use of typed addresses rather 
than labels or envelopes with 
windows; 

4) the use of a pre-addressed envelope 
rather than a business reply enve- 
lope which is often associated with 
householder/ junk mail. 


If the subject matter being studied is 
of a technical or work-oriented nature, 
the outgoing letter should be ad- 
dressed to the respondent’s work ad- 
dress. Questionnaires asking for 
personal information should be ad- 
dressed to the respondent’s home 
address. Whether or not the respon- 


dent should be named in the address 
depends on the degree of personaliza- 
tion required and whether or not sub- 
stitution of respondents will affect 
the study. 


The day of mailing of the outgoing 
letter does not affect the overall rate 
of response. 


6.FORMAT & APPEARANCE 


Generally, the response rates to 
printed questions were not signifi- 
cantly different from the response 
rates to the same questions which were 
mimeographed. Letterhead stationary 
does not necessarily guarantee a sig- 
nificantly higher response rate over 
non-letterhead. However, if a letter- 
head is to be used, it has been estab- 
lished that the letterheads of some 
organizations (for example, govern- 
ments) do elicit better response rates 
than others. 


The few studies that have dealt with 
the effect of coloured vs. white paper 
have not established colour as a factor 
which influences response rate. 


6.1 LENGTH OF QUESTIONNAIRE 


Peripheral questions which gather in- 
formation useful to the study need not 
be sacrificed in order to have a short 
questionnaire. Although common 
sense dictates that the questionnaire be | 
as brief as possible, most studies prove 
that there is no significant difference 
in the response rates for short and 
long questionnaires. In fact, the addi- 
tion of interesting questions to a ques- 
tionnaire has improved the response 
rate. 


It would appear that a high response 
rate is more dependent on the interest 
that the questionnaire generates than 
on its brevity. 


6.2 PRE-CODING — 


Whether or not responses to questions 
should be pre-coded depends on the 
subject matter and objectives of the 
study. For example, explorative ques- 
tionnaires with open-ended questions 
are not easily pre-coded. Precoding to 
facilitate the process of transcribing 
data from the questionnaire toa 
machine-readable medium or to facili- 
tate manual tabulation does not ad- 
versely affect the response rate to mail 
questionnaires. 


Although pre-coding has not been 
shown to increase response rates, it is 
an important consideration in ques- 
tionnaire design in light of its contribu- 
tion to processing the data and is 
therefore mentioned here. 


7. FOLLOW-UPS 


Any contact with respondents after 
the initial mail out of the question- 
naire to prompt or remind them to 
complete and return the questionnaire 
is regarded as a follow-up. Contact 
can take the form of letters, post- 
cards, telephone calls, personal visits 
or some combination of these. They 
can be directed at the whole sample 
or only at those who have not yet re- 
turned the questionnaire (if they can 
be identified). 


One technique which seems to be con- 
sistently effective in increasing re- 
sponse rates is follow-up mail. Post- 


cards or letters are sent several days 
after the original questionnaires have 
been mailed out. 


If possible, an additional copy of the 
questionnaire should be enclosed, since 
the first may have been lost or mislaid. 


The use of follow-up is generally ac- 
cepted as the most effective method to 
increase response rates and it is there- 
fore recommended that at least one 

or two be conducted for each survey. 
Since each successive follow-up results 
in added returns, there is the potential 
to achieve an extremely high total re- 
sponse rate. However, the value of 
additional information thus obtained 
must be weighed against the costs and 
time required for successive contacts. 


8. METHODOLOGY 


A number of studies in recent years 
have been concerned with increasing 
response rates and reducing biases in 
mail surveys. The first seven sections 
of this booklet have dealt with some of 
the methods that have been successful 
in increasing response rates. This last 
section will identify a few of the prob- 
lems associated with non-response 
and will indicate some methods avail- 
able for weighting findings so that the 
characteristics of the population un- 
der study can be more accurately 
reflected. 


8.1 NON-RESPONSE BIAS 


A major problem common to mail 
surveys is a low response rate. When 
confronted with this problem, the 
researcher is forced to be concerned 
with the bias that could be introduced 
as a result of systematic differences 


between those who respond and those 
who do not. 


Non-respondents may be classified into 
two major groups — those who do not 
receive the material and those who 
decline to respond. The size of the 
former group is a function of the 
recency and accuracy of the sampling 
frame. The size of the latter group 
may be a function of the subject 
matter of the survey, any of the 
methods used in that particular survey 
or the individual’s general attitudes 
towards survey research. 


In theory, the degree of non-response 
bias in a survey is directly related to 
the proportion of non-respondents in 
the total sample, and the extent of 
difference between respondents and 
non-respondents on the variables in- 
vestigated. Therefore, minimizing 
non-response bias requires a reduction 
in the non-response proportion as well 
as information on the characteristics 
of non-respondents. 


The measurement of non-response 
bias is very difficult since the variables 
needed for analysis depend on data 
which the non-respondents, by defini- 
tion, fail to make available. The lite- 
rature on non-response bias describes 
four methods for estimating popula- 
tion values: 


(1) comparisons with “‘known”’ values 
of the population (such as age, 
income). This may reveal the 
extent to which the sample is or is 
not representative of the popula- 
tion on the variables of interest. 


(2) estimates of population values by 
extrapolating trends across re- 
sponse waves. This is based on the 
assumption that subjects who 
respond later are more like non- 
respondents. If the researcher 


assumes that the last wave or a 
combination of the last waves are 
representative of all the non- 
respondents to the first mailing, 
then the final non-response is 
weighted by replies to the follow- 
up mailings. 


(3) weighting the actual responses ob- 
tained so as to represent the total 
sample. The assumption made here 
is that there is no difference be- 
tween the characteristics of those 
who respond and those who do 
not. The weight for each response 
is given by the total sample size 
divided by the number of respon- 
dents. 


(4) minimizing non-response bias by 
reducing non-response itself. This 
may be achieved by extensive 
follow-up of a sub-sample of non- 
respondents. 

Although the mail survey has been 
mainly criticized for non-response 
bias, some of the other sources of 
bias in mail surveys are: 


(1) imperfect design of the 
questionnaire; 

(2) the responses made by a person 
other than the one to whom 
the questionnaire was ad- 
dressed; 

(3) group responses when an indi- 
vidual response is required; 

(4) proxy response i.e. parents 
responding for children; and, 

(5) late returns. 

The direction of research efforts 

to determine differences between 

respondents and non-respondents 
has been focussed mainly on de- 
mographic, socio-economic and to 
some extent on personality vari- 
ables. The only widespread 
finding has been that on an aver- 
age, respondents tend to be better 
educated than non-respondents. 


8.2 EFFECT OF INCOMPLETE 
QUESTIONNAIRES 


In addition to response rate and 
response speed, response completeness 
is another response pattern which 
effects the analysis of data. Incomple- 
teness of responses on an individual 
questionnaire will extend the overall 
non-response problem for the omitted 
items and will hinder cross-compari- 
son of variables at the analysis stage. 


Many theories have evolved to explain 
why people do not fully answer 
questionnaires. The most common 
reasons presented are: lack of interest 
in the subject; the unresponsive mood 
of some recipients, lack of under- 
standing of the questions asked; sus- 
picion of intentions; poor question- 
naire design; and the mental and phys- 
ical chore involved. This can result in 
the production of incomplete responses 
which will introduce response bias 
since those who respond to particular 
questions may possess different per- 
sonal characteristics and attitudes 
than the non-respondents of these 
questions. 


Even with the best sample design, the 
presence of incomplete response may 
cause errors in estimates that depict 
the population investigated and may 
result in predictions which are inac- 
curate and misleading. 


8.3 RELIABILITY OF RESULTS 


Efforts expended in achieving an un- 
biased sampling of respondents will be 
wasted unless one can be confident 
that the responses given by individuals 
correctly represent the facts and 
attitudes which the questions are de- 
signed to elicit. This can be measured 


by comparing the responses with 
known facts. However, when this is 
not possible, two administrations of 
similar questionnaires can be com- 
pared to obtain a measure of reli- 
ability. Unfortunately, literature re- 
porting on mail surveys provides very 
little information on the reliability of 
mail survey responses in general. 
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